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ABSTRACT 

This study outlines and evaluates the admissions 
policies and procedures devised for evaluating applicants to the 
Indiana Academy for Science, Mathematics, and Humanities. The 
procedures utilized a holistic file review process that required 
applications to be evaluated in the context of their local regions. 
Complete student files, which included standardized test scores, 
grade transcripts, teacher and administrator recommendations, and 
student accounts of accomplishments, were evaluated and assigned a 
score in aptitude, achievement, and accomplishment. The file review 
process determined who would be invited to interview for admission. 
In the first 2 years of the Academy, 160 students were invited to 
interview. Results of statistical analysis of student records showed 
that: (1) the admissions procedures produced a student body that was 
characterized by academic excellence, geographic diversity, and 
ethnographic diversity; (2) the aptitude, achievement, 
accomplishment, and composite ratings from the holistic method were 
positively correlated with usual predictors such as Scholastic 
Aptitude Test scores, grades, and teacher recommendations and were 
positively correlated with academy grade point averages; and (3) 
canonical correlation and stepwise regression procedures do not 
produce results that would lead to the construction of a useful 
analytical model for predicting academic performance. These findings 
support the continued use of the holistic file review process and 
demonstrate that a holistic student identification method encourages 
attention to equity issues that usual predictors do not. (Contains 18 
references .) (JDD) 
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Evaluation of a Holistic Method for Identifying Pupils 
for State Academies for the Academically Gifted 

Abstract 

Identifying pupils for admission to state academies for the 
academically gifted is enigmatic. Assuring equity of educational 
opportunity is as important as identifying pupils with the 
greatest potential to succeed in an academically rigorous 
environment. When the Indiana Academy for Science, Mathematics, 
and Humanities planned its admissions policies and procedures, it 
devised a holistic method for evaluating applicants. In this 
study the procedures used by the Indiana Academy are outlined and 
evaluated. Results show that ratings from a holistic method for 
evaluating student applications are positively correlated with 
usual predictors such as SAT scores, grades, and teacher 
recommendations. Moreover, a holistic student identification 
methoa encourages attention to equity issues that usual predictors 
do not. The experience of the Indiana Academy's first two years 
supports continued use of holistic methods for identifying 
students for admission. 



Evaluation of a Holistic Method for Identifying Pupils 
for State Academies for the Academically Gifted 

Although barely over a decade old, the movement to establish 
state supported residential academies for students who are 
intellectually gifted continues to grow. The potential benefits 
espoused by the founders of these schools are beginning to be 
evidenced, with graduates now assuming adult responsibilities and 
outreach programs becoming widespread. 

Among the lingering concerns, however, is hew state academies 
admit students. Identifying students for gifted education 
programs has always been enigmatic, and state academies have not 
discovered any immunity from the same perplexing problems that 
face educators in other schools. Ambiguous conceptions of 
giftedness, misuse of tests, validity and reliability of non- 
standard sources cf information, and fear of elitism are major 
apprehensions as state academies design or review admission 
policies and procedures. Moreover, state academies for the gifted 
must function in a political context. Equity issues are vital, as 
is the obvious purpose of identifying students who have the 
potential to succeed in a highly enriched educational environment. 
Facing these challenges, it came as no surprise that one state 
academy--the Indiana Academy for Science, Mathematics, and 
Humanit ies--gr ve careful attention to the design and 
implementation of its admission procedures when it planned its 
inaugural year. 
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In this article two aspects of student selection in the 
Indiana Academy are examined. The method itself — a holistic file 
review process — is explained. In addition, a statistical analysis 
of the results of the Indiana Academy's admissions policies and 
procedures for its first two years is presented. Two questions 
guided the research. First, can equity and excellence be 
compatible in student selection process for the academically 
gifted? And second, does the process of evaluating students 1 
applications for the purpose of admission have predictive value 
for academic achievement? 

Review of the Literature 
Identifying pupils for gifted programs of any type is fraught 

with fallacies. Both the National Report on Identification 

(Richert, Alvino, & McDonnel, 1982) and the Richardson report 
(Cox, Daniel, & Boston, 1985) cited many of the faulty 
identification practices prevalent in schools. Among some of the 
more obvious problems, according to Richert (1987) , include 
elitist or distorted definitions of giftedness, confusion about 
the purposes of identification, misuse and abuse of tests, and 
inappropriate use of multiple criteria. 

In an attempt to close the gap between what we know and what 
we practice, Richert (1987) has recommended several research based 
guidelines for identification. 

First , the definitions should reflect current theories 
(Richert, 1987) . Accordingly, definitions of giftedness that are 
going to be used for identification should be broad and inclusive 
so that they acknowledge the multi-faceted characteristics of 



giftedness. Gardner (1985), Sternberg, (1985), and Renzulli 
(1986) all argue toward a broadened definition. 

Second, test data must be used appropriately. Richert et al 
(1982) have documented many erroneous uses of tests for 
identifying the gifted. The key is ensuring that any test usod 
has been established through research as appropriate for the 
ability being sought. The Scholastic Aptitude T^st (SAT), 
although controversial, has been proven to be an effective 
predictor of scholastic aptitude for the academically gifted and 
talented at the middle school and secondary levels (Stanley, 
1976) . Richert and her colleagues (1982) , however, argue against 
the use of cut-off scores on tests, since other reliable sources 
of data, such as grades or actual performance, also can be made 
readily available. Also essential is determining whether a test 
is known to discriminate against any of the disadvantaged 
populations (Richert et al, 1982) . 

If discrimination is to be avoided in an identification 
system, special procedures will be required. Richert and her 
colleagues (1982) identified students with low socio-economic 
status, minority races or cultures, students with minimal 
proficiency in English, and females (when identifying ability in 
mathematics) as most likely to be overlooked when tests were 
emphasized. 

The National Report on Identification stated that appropriate 
use of multiple sources of data was necessary for an 
identification system to be equitable and inclusive. Several 
precautions should be taken, however, when using multiple sources 




of data, warned Richert and her colleagues. Schools should avoid 
inserting data from different sources into a formula. The purpose 
of using multiple sources is not to validate one source with 
another; but rather, it is to have a variety of sources that 
complement one another so that the several indicators can reveal- 
something that a single indicator can not. Alternative sources of 
information include statements by parents, ratings and written 
recommendations from teachers, peer nominations, self-nominations, 
and interviews (Richert, 1987) . 

Although recommendations for practice in the Nationa l Report 
on Irient j f icat ion include the appropriate use of multiple sources 
of data, research on specific sources suggests that no single 
source be given great weight by itself. Hagen (1980) reported 
that research has yet to establish either the efficiency or 
effectiveness of teacher rating scales. Students with negative 
behavior traits, limited language backgrounds, and handicapped 
students are especially vulnerable to being overlooked by 
teachers. Hagen (1980) also supported the use of biographical 
inventories, noting that they had a higher correlation between 
creative achievement than did tests of creativity. But, she 
argued against the use of personality inventories and attitude 
scales since their intended uses are clinical. 

Information on the accomplishments of students outside the 
classroom should be given strong consideration. Wallach (1976) 
reported that tests scores and grades hold limited ability to 
predict adult accomplishment, whereas Mundy and Davis (1974) found 

4 

ERLC / 



that adult accomplishments were related to comparable high school 
non-academic (extra-curricular) accomplishments . 

Interviews with students are most frequently used in the 
assessment stage after a talent pool has been found, according to 
Richert (1987) . Myslenski and Jeffery (1985) found that 
interviews are, at best, subjective, and empirical support for 
their use as a valid method of predicting success in academic 
programs is lacking. 

Published reports on the effectiveness of specific 
identification systems is scant. In 1987 Bunch and Scherick 
studied data used in the evaluation of students 1 applications for 
admission to the North Carolina School for Science and 
Mathematics, comparing the data to academic achievement after 
enrollment in the school . Among their significant findings was 
the high correlation between the SAT verbal score and grade point 
average . 

Overview of Indiana Academy Admissions Practices 

Admissions to the Indiana Academy during the two year period 
studied was highly selective. Based on multiple criteria and 
multiple sources of data, the application evaluation process 
utilized a holistic, file review process (Green, 1990). 
Mathematical reasoning, scientific reasoning, verbal reasoning, 
interpersonal ability, and performance ability were included in 
the evaluation process. Applicants supplied SAT scores taken 
during their sophomore year, transcripts of class grades from the 
seventh grade through the first semester of the sophomore year, 
and recommendations from an English teacher, mathematics teacher, 
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science teacher, and either a principal, guidance counselor, or 
gifted education program director. The recommendations were 
written on academy forms so that responses to specific questions 
provided the academy's file reviewers with anecdotal information 
depicting creativity and problem solving ability, as well as other 
relevant attributes researchers associate with academic 
giftedness. Students themselves provided detailed accounts of 
their accomplishments and awards in order to indicate their 
qualities of motivation and self-discipline in addition to 
revealing their special talents. 

Complete student files were evaluated by a team of file 
reviewers, with three persons reviewing each file. For each file 
reviewed, at least one of the reviewers came from the region where 
the student resided so that the local context of the student could 
be considered when evaluating the information. Each file reviewer 
assigned a holistic score (ranging from 20 to 80) in each of three 
areas: aptitude, achievement, and accomplishment. Also, each 
reviewer assigned a composite file review score. The composite 
file review score could not be less than nor greater than any of 
the three separate sub-scores. The three file review scores from 
the three individual evaluators were then added, and the sum 
became the final file review score for the application. 

The process took great care to see that scores from the 
reviewers were reliable. If the three file reviewers 1 scores 
deviated significantly, then additional review of the file was 
required. All the persons who served as reviewers were trained by 
Indiana Academy personnel (Freyberg, 1990) . 



The state was divided into 15 zones with tenth grade 
populations of approximately equal size. The students were ranked 
according to their file review scores, and then were invited 
according to their standing in their respective geographic zones. 
In the first two years of the Indiana Academy, 160 students were 
invited. The first 105 invited were drawn from the 15 zones, 
seven from each zone. The final 55 students invited came from "at 
large." As a result of inviting the at large applicants after the 
applicants from the geographic zones had been identified, the 
possibility that a student invited from a zone might have a final 
file review score lower than a student not invited was remote. 
Also in this fashion, geographic distribution across the state was 
optimized. And, because the ethnic diversity of the state has 
definite geographic characteristics, ethnic diversity was 
positively influenced . 

The file review process determined who would be invited to 
interview for admission to the academy. The interview was placed 
at the end of the process so that it would occur after a student 
was deemed qualified. The interview itself was not designed to 
evaluate a student's potential giftedness. But rather, it was an 
opportunity for academy officials to make certain that the 
expectations of the academy were made clear to the candidate 
before the candidate decided to accept the invitation to enroll. 
The academy reserved the right to withhold the admissions decision 
until after the interview. 
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Research Procedure 

Two central questions emerge from any discussion of the 
admissions policies and procedures of a state academy. First, do 
the procedures provide for social equity as well as identify 
academic giftedness? And second, do the procedures have 
predictive value? That is, do the procedures identify those 
students who are most likely to be successful in the educational 
program? In order to answer these questions, two stages of 
research were undertaken. 

In response to the first question, a demographic analysis of 
the students who enrolled in the Indiana Academy was conducted to 
determine whether traditionally underrepresented populations were 
sufficiently represented in the first two classes. 

In response to whether the file review procedures had 
predictive value, correlations were calculated among the usual 
predictors vi.e., SAT-verbal; SAT-quantitative; TSWE; and 
recommendations from science, mathematics, and English teachers, 
and administrators) . Next correlations were calculated among the 
academy's ratings (i.e., holistic ratings for "aptitude," 
"achievement," and "accomplishment"). These correlations were to 
allow the investigator to observe the reliability of the various 
predictors. Correlations were also calculated between the usual 
predictors and the academy's ratings to allow the observation of 
relationships existing between the two sets of variables. 

The subsequent analysis consisted of the calculation of 
canonical correlations for the two sets of variables (usual 
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predictors and academy ratings) and academy grade point averages 
in three areas: science, mathematics, and humanities. 

Finally, stepwise multiple regression was performed in order 
to construct a predictor model for possible use in the future. 

Admissions variables defined for the analyses included SAT 
(verbal), SAT (Quantitative), TSWE, GPA (grades 7 - 10.5 in core 
subjects), English teacher ratings, science teacher ratings, 
mathematics teacher ratings , administrator ratings, holistic 
rating for accomplishment, holistic rating for achievement, 
holistic rating for aptitude, and the final holistic file review 
score. Each variable was correlated with separate grade point 
averages: one for grades in science, one for grades in 
mathematics, and one for grades in humanities. Since the Indiana 
Academy does not maintain grade point averages for students (since 
it does not maintain class rank) , grade point averages were 
calculated expressly for this study. The academy's grading scale 
of A,B,C, and Unsatisfactory allowed the use of a 3.00 continuous 
scale. Grades in certain mathematics and science courses were 
weighted since the curriculum expressly addressed levels of 
abstraction and acceleration in those subject areas. Table 1 
contains averages for all the variables included in the study. 



TABLE 1 

MEANS AND STANDARD DEVIATION 
FOR ADMISSIONS VARIABLES 
(N = 500) 



VARIABLE 


MEAN 


STANDARD 
DEVIATION 


SAT VERBAL 


505.82 


85.33 


SAT QUANTITATIVE 


576.42 


97.97 


TSWE 


50.96 


7.29 


GPA RECALCULATED 


3.64 


0.39 


SCIENCETEACHER RATING 


4.61 


0.97 


MATH TPAPWPR RATIMf5 
IW\I rl I CrtOntn nnl UNO 


*+ • o o 


1 02 


ENGUSH TEACHER RATING 


4.60 


1.05 


ACCOMPLISHMENT RATING 


54.63 


1 1 .21 


ACHIEVEMENT RATING 


50.95 


16.25 


APTITUDE RATING 


50.53 


13.79 


FILE REVIEW SCORE 


158.44 


38.48 


Profile of the 


First Two 


Classes 



The results in the first two years revealed that the first 
two classes were distinguished by their academic qualifications 
and characterized by their diversity (Green, 1991) . The mean 
composite SAT score for students enrolled in the first year was 
1140 and the mean TSWE was 54 (IASMH Profile Enrolling Juniors, 
1991) . In the second year the mean composite SAT was 1130 and the 
mean TSWE was 52 (IASMH Class of 1993 Profile, 1992) . And the 
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average cumulative grade point average at the time of admission 
was 3.90 on a 4.00 scale for the first class, and 3.80 for the 
second class. In both of the first two classes, over a quarter of 
the students enrolling ranked first in their respective home 
schools when they enrolled in the academy, and their average class 
rank was fourth. Of the 144 students in the inaugural class who 
took the PSAT in October, 1990 (the inaugural year) , 29 were 
recognized as National Merit Scholarship semi -f inalis t s , 
representing 20 per cent of the graduating class and 7 per cent of 
all Nati )nal Merit semi-finalists from Indiana (IASMH School 
Profile, 1991) . 

Enrollment: Equity 

Moreover, the classes for the first two years represented all 
of Indiana. In the first two years of admissions experience, 75 
of Indiana's 92 counties have been represented, and 169 of the 
state's high schools. In the first year, 49 per cent of the class 
was female and 51 per cent was male. In the second year, 59 per 
cent of the incoming class was female and 41 per cent was male. 
In the first class, over 13 per cent of the students came from 
minority populations, and in the second class the percentage 
exceeded 17 per cent--proport ions that exceeded the minority 
population of the comparable state-wide age group (IASMH Profile 
Enrolling Juniors, 1991; IASMH Class of 1993 Profile, 1992.) 

The academic and demographic characteristics of the first two 
classes provide convincing evidence that academic excellence and 
demographic equity can be achieved simultaneously. 
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Academic Performance 



The academic performance of the inaugural class proved to be 
quite satisfactory in a curriculum designed to accelerate and 
intensify the study of traditional liberal arts and sciences. 
When the inaugural class graduated, over three fourths of the 
students earned grades that were classified as "Outstanding" or 
"Superior." Five per cent or less of the grades were 
"Unsatisfactory." Tables 2 through 4 contain the grade 
distributions for the inaugural class 1 two years at the Indiana 
Academy . 



Humanities Grade Distribution of the Class of 1992 of the 
Indiana Academy for Science, Mathematics, and Humanities 



Table 2 



Humanities 



90-91 



91-92 



English : 



A-Outstanding 
B-Superior 
C-Sat is factory 
U-Unsat is factory 



52% 
34% 
12% 
2% 



47% 
49% 
4% 
0% 



History : 



A-Out standing 
B-Superior 
C-Sat is factory 
U-Unsat is factory 



48% 
46% 
6% 
0% 



46% 
43% 
10% 
0% 



Foreign Language : 



A-Out standing 
B-Superior 
C-Sat is factory 
U-Unsat is factory 



52% 
32% 
11% 
5% 



52% 
30% 
15% 
3% 



12 

1 cr 



Table 3 



Science Grade Distribution for the Class of 1992 of the 
Indiana Academy for Science, Mathematics, and Humanities 



90-91 91-92 

A-Outstanding 49% 68% 

B-Superior 38% 22% 

C-Satisfactory 13% 8% 

U-Unsatisfactory 0% 1% 



Table 4 

Mathematics Grade Distribution for the Class of 1992 of 
the Indiana Academy for Science, Mathematics, and 

Humanities 



90-91 91-92 

A-Outstanding 35% 2 9% 

B-Superior 40% 38% 

C-Satisfactory 23% 28% 

U-Unsatisfactory 2% 4% 



Results of Research: 
Relationship of Usual Predictors to Academy Ratings 



Procedures for determining whether the academy's file review 
proc* ss have value for predicting academic success began with an 
analysis of the relationship of the usual predictors to ratings 
used by the academy. Usual predictors included the SAT (verbal) ; 
SAT (quantitative); Test of Standard Written English (TSWE) ; grade 
point average (GPA) ; and the average rating on recommendation 
forms for English teacher, mathematics teacher, and science 
teacher. The GPA was recalculated by the academy so that it only 
included academic subjects (English, mathematics, science, social 
studies, foreign languages, computer literacy) and included grades 
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seven through ten (first semester) . The academy ratings included 
holistic scores for aptitude, achievement, accomplishment, and a 
holistic composite score — the file review score. A total of seven 
usual predictors and four academy ratings were included in the 
analysis, with a total of 500 observations available for the 
analysis. Table 1 contains the list of the variables with their 
mean values and standard deviations. 

First, correlations among the usual predictors were 
calculated. Table 5 contains the correlation statistics. As 
expected, positive correlations among the SAT-verbal, SAT- 
quantitative, and TSWE were high (r > .50) . The correlations for 
the teacher recommendations also were positive, but notably weak. 
Simple observation of these correlations corroborates that 
standardized tests are more reliable than teacher recommendations 
on forms used to identify academic giftedness. 



TABLE 5 

CORRELATIONS AMONG THE USUAL PREDICTORS 
(N = 500) 





SAT 
VERBAL 


SAT 
MATH 


TSWE 


GPA 
RECALCULATED 


SCENCE 
TEACHER 
RATING 


MATH 
TEACHER 
RATING 


ENGUSH 
TEACHER 
RATING 


SAT VERBAL 


1 .00 


0.52 


0.65 


0.29 


0.20 


0.1 1 


0.18 


SAT MATH 


0.52 


1 .00 


0.52 


0.36 


0.21 


0.20 


0.18 


TSWE 


0.65 


0.52 


1 .00 


0.39 


0.20 


0.15 


0.20 


GPA RECALCULATED 


0.29 


0.36 


0.39 


1.00 


0.33 


0.33 


0.32 


SCIENCE TEACHER RATING 


0.20 


0.21 


0.20 


0.33 


1 .00 


0.30 


0.38 


MATH TEACHER RATING 


0.1 1 


0.20 


0.15 


0.33 


0.30 


1 .00 


0.31 


ENGLISH TEACHER RATING 


0.18 


0.18 


0.20 


0.32 


0.38 


0.31 


1 .00 



Next, correlations among the academy ratings were calculated. 
Table 6 contains the correlation statistics. Notably, all the 
correlations were positive (r > .60 in all cases), indicating the 
file review process produced reliable ratings for the factors of 
aptitude, achievement, and accomplishment . 

TABLE 6 

CORRELATIONS AMONG ACADEMY RATINGS 
(N = 500) 





ACCOMPLISHMENT 
RATING 


ACHIEVEMENT 
RATING 


APTITUDE 
RATING 


FILE 
REVIEW 
SCORE 


ACCOMPLISHMENT RATING 


1.00 


0.72 


0.65 


0.91 


ACHIEVEMENT RATING 


0.72 


1.00 


0.61 


0.84 


APTITUDE RATING 


0.65 


0.61 


1 .00 


0.83 


FILE REVIEW SCORE 


0.91 


0.84 


0.83 


1 .00 



Correlations between the usual predictors and the academy 
ratings were also calculated. (See Table 7.) Positive 
correlations were noticeable between accomplishment and GPA (r = 
.50); accomplishment and English teacher recommendation (r = .51); 
and accomplishment and science teacher recommendation (r = .56). 
Positive correlation was strong between achievement and GPA (r = 
.74) . Positive correlations were strong between aptitude and SAT- 
verbal (r = .74); aptitude and SAT-quantitative (r = .72); and 
aptitude and TSWE (r = .67) . Most importantly, positive 
correlations were noticeable between the file review score (the 
composite holistic academy rating) and all the usual predictors. 



The correlations between the file review scores and the SAT-verbal 
scores, the SAT-quantitative scores, the TSWE's, and the CPA's 
were r = .55, r = .57, r = .55, and r = .60 respectively. These 
positive correlations establish a strong positive relationship 
between Indiana Academy's holistic file review process and the 
usual prediction measures. 

TABLE 7 

CORRELATIONS AMONG THE USUAL PREDICTOR 
AND ACADEMY RATINGS 
(N = 500) 





ACCOMPLISHMENT 
RATING 


ACHIEVEMENT 
RATING 


APTITUDE 
RATING 


FILE 
REVIEW 
SCORE 


SAT VERBAL 


0.38 


0.36 


0.74 


0.55 


SAT MATH 


0.44 


0.40 


0.72 


0.57 


TSWE 


0.39 


0.41 


0.67 


0.55 


GPA RECALCULATED 


0.50 


0.74 


0.40 


0.61 


SCIENCE TEACHER RATING 


0.56 


0.45 


0.31 


0.51 


MATH TEACHER RATING 


0.43 


0.36 


0.20 


0.36 


ENGLISH TEACHER RATING 


0.51 


0.42 


0.28 


0.46 



In addition, canonical correlation coefficients were 
calculated for the usual predictors and the academy ratings. 
Table 8 contains the standardized canonical correlation 
coefficients for the usual predictors and the academy ratings. 
Inspection of these coefficients reveals three important 
observations. First, students who scored high on both the SAT- 
quantitative and SAT-verbal tended to have high academy ratings 



for aptitude and, to a lesser degree, a high academy file review 
score. Second, students who had a high recalculated GPA and who 
scored low on the SAT-verbal tended to have high academy 
achievement ratings and low academy aptitude ratings. Third, 
students who had low recalculated GPA ? s and high recommendations 
from science, mathematics, and English teachers tended to have 
high academy accomplishment ratings, but low academy achievement 
ratings and low academy file review scores. These canonical 
correlations tend to validate the academy's holistic rating 
system. 

TABLE 8 

STANDARDIZED CANONICAL COEFFICIENTS FOR THE 
USUAL PREDICTORS AND THE ACADEMY RATINGS 
(N = 500) 





USUAL 


PREDICTORS 






SAT VERBAL 


0.42 


-0.35 


-0.08 


0.06 


SAT MATH 


0.41 


-0.29 


0.15 


-0.65 


TSWE 


0.20 


-0.16 


-0.05 


0.57 


GPA RECALCULATED 


0.13 


0.76 


-0.88 


-0.07 


SCIENCE TEACHER RATING 


0.12 


0.29 


0.51 


0.38 


MATH TEACHER RATING 


0.01 


0.16 


0.38 


-0.73 


ENGLISH TEACHER RATING 


0.10 


0.20 


0.42 


0.35 




ACADEMY RATINGS 






ACCOMPLISHMENT RATING 


-0.1 1 


0.48 


2.02 


-2.00 


ACHIEVEMENT RATING 


0.03 


0.97 


-0.79 


-1.63 


APTITUDE RATING 


0.78 


-0.97 


0.15 


-1.76 


FILE REVIEW SCORE 


0.32 


-0.05 


-1.16 


4.8?. 



9 

ERJC 



17 



Results of Research: 
Relationship of Admissions Variables to Academic 

Performance 

For the purpose of this study, academic performance was 
defined in terms of semester grades earned. As previously noted, 
the Indiana Academy did not report grade point averages because 
class rank was not considered a relevant statistic in a population 
comprised exclusively of academically gifted students. However, 
for this investigation grade point averages were calculated for 
the three areas in the curriculum: science, mathematics, and 
humanities. Both classes were included in the study. Weighted 
grades were used in mathematics and science because in those 
subject areas the level of abstraction and acceleration were 
distinguished in the design of the curriculum. 

Observation of the correlations between the us ial predictors 
and academy ratings and the academy grade point averages was 
revealing. There were six instances of positive correlations 
between the holistic academy ratings and academy GPA's exceeding 
.40 (achievement rating and math GPA, file review score and math 
GPA, achievement rating and science GPA, file review score and 
science GPA, achievement rating and humanities GPA, and file 
re\ iew score and humanities GPA). However, there was only one 
instance of a positive correlation between a usual predictor and 
academy GPA's exceeding .40— the recalculated GPA from previous 
schools and academy math GPA. Table 9 contains correlations among 
GPA's for science, mathematics, and humanities areas. Table 10 
contains all the correlations for the 12 original variables and 
academy GPA's. 
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TABLE 9 



CORRELATIONS AMONG THE ACADEMY GPA'S 

MATHGPA SCIENCE GPA HUMANITIES GPA 
MATHGPA 1.00 0.69 0.57 
SCIENCEGPA 0.69 1.00 0.57 
HUMANITIES GPA 0.57 0.57 1.00 



TABLE 10 

CORRELATIONS AMONG THE HS & ACADEMY PREDICTORS &THE ACADEMY GPA'S 

MATHGPA SCIENCEGPA HUMANITIES GPA 

SAT VERBAL 0.23 0.29 0.27 

SAT MATH 0.33 0.40 0.05 

TSWE 0.23 0.32 0.33 

GPA RECALCULATED 0.41 0.37 0.38 

SCIENCE TEACHER RATING 0.23 0.17 0.24 

MATH TEACHER RATING 0.14 0.08 0.14 

ENGLISH TEACHER RATING 0.25 0.26 0.36 

ADMINISTRATION RATING 0.23 0.21 0.24 

ACCOMPLISHMENT RATING 0.29 0.32 0.35 

ACHIEVE MENT RATING 0.42 0.40 0.43 

APTITUDE RATING 0.28 0.35 0.20 

FILE REVIEW SCORE 0.38 0.44 0.40 



19 



ERIC 2 



ERIC 



In addition, canonical correlation coefficients were 
calculated for the academy GPA's and 12 original admissions 
variables. Table 11 contains the standardized canonical 
coefficients for the 12 original admissions predictors and academy 
GPA ? s. Analysis of Table 11 reveals two important findings. 
First, students who had high academy achievement ratings, high SAT- 
verbal scores, and high English teacher recommendations tended to 
have high GPA's in humanities; and, to a lesser extent, they had 
high GPA's in science and mathematics. Second, students who had 
low SAT-quantitative scores but had high TSWE scores tended to 
have high GPA's in humanities and low GPA's in both science and 
mathematics . 
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TABLE 11 



CANONICAL CORRELATION COEFFICIENTS 
FOR USUAL AND ACADEMY PREDICTORS 
AND ACADEMY GPA'S 
(N = 255) 



SAT VERBAL 
SAT MATH 
TSWE 

GPA RECALCULATED 
SCIENCE TEACHER RATING 
MATH TEACHER RATING 
ENGLISH TEACHER RATING 
ADMINISTRATION RATING 

ACCOMPLISHMENT RATING 
ACHIEVEMENT RATING 
APTITUDE RATING 
FILE REVIEW SCORE 

MATH GPA 
SCIENCE GPA 
HUMANITIES GPA 



USUAL PREDICTORS 

0.29 0.18 

0.20 -1.02 

0.21 0.47 

0.21 -0.24 

0.06 -0.00 

-0.06 0.10 

0.27 0.11 

0.02 0.17 
ACADEMY RATINGS 

0.21 0.07 

0.35 0.13 

-0.26 -0.23 

0.01 0.09 
GRADE POINT AVERAGES 

0.25 -0.54 

0.39 -0.69 

0.52 1.16 



0.02 
0.16 
0.56 
-0.71 
-0.47 
-0.31 
0.24 
-0.12 

-0.42 
-0.16 
-0.56 
1.23 

-1 .31 
1.21 
0.03 




The canonical correlations among the original admission 
variables and the academy GPA's tend to suggest that the academy's 
holistic rating system for achievement, combined with SAT-verbal 
scores, TSWE scores, and the English teacher recommendation, is a 
valid predictor of academic performance in the academy. 

Results of Research: 
Model for Predicting Academic Success in the Academy 

Since the canonical correlations among the original 

admissions variables and the academy GPA's suggested that the 

academy's holistic achievement rating, the SAT-verbal score, the 

TSWE, and the English teacher recommendation all have value for 

predicting grade point averages in the academy, the stepwise 

regression procedure was performed, using the composite academy 

GPA as the dependent variable. The analysis was conducted for the 

purpose of determining whether an academic performance prediction 

model could be constructed. 

In the first step of the procedure, when the academy's 

holistic achievement rating was entered into the formula, 24% of 

the variance of the composite GPA was explained by the academy's 

holistic achievement rating. In the second step of the procedure 

the TSWE variable was entered, and an additional 4% of the 

variance was produced. In the third step of the procedure the 

English teacher recommendation was entered into the formula, and 

an addi + ional 4% of the variance was produced. In the fourth and 

final step of the procedure, the variable for SAT-verbal score was 

entered in the formula, which added another 1% to the explanation 

for the variance. All in all, only 33% of the variance of the 



22 



compos/.te GPA was explained by the academy's holistic achievement 
rating, TSWE, English teacher recommendation, and SAT-verbal 
score. The summary of the stepwise regression procedure is 
contained in Table 12 . 

TABLE 12 

SUMMARY OF STEPWISE PROCEDURE FOR COMPOSITE GPA 



STEP 


VARIABLE ENTERED 


NUMBER 


PARTIAL R2 


MODEL R2 


F 


PROBABILITY > F 


1 


ACHIEVEMENT RATING 


1 


0.24 


0.24 


79.96 


0.00 


2 


73 WE 


2 


0.04 


0.28 


14.24 


0.00 


3 


ENGLISH RATING 


3 


0.04 


0.32 


13.57 


0.00 


4 


SAT VERBAL 


4 


0.01 


0.33 


5.09 


0.03 



While the results of the stepwise regression procedure 
indicated that a noticeable proportion of the variance of 
composite GPA in the Indiana Academy were explained by the 
academy's holistic achievement rating, TSWE, English teacher 
recommendation, and SAT-verbal score, the formula would not be 
useful as a model for predicting academic performance in the 
academy. Other factors, undefined by statistical models, 
contributed more to variance of composite GPA's. 



Summary and Conclusions 

Identifying students for admission to state academies for the 
academically gifted remains enigmatic . Students must be selected 
on the basis of their potential for succeeding in an academically 
rigorous environment; but, the only means we have for predicting 
future performance is observing past and present performance. 
Many educators, however, are aware that traditional methods for 
observing past and present performance — such as standardized tests 
and previous grades — are inadequate by themselves for identifying 
academically gifted students. The issues of gender and race bias 
are well known, and the tests themselves are one dimensional — they 
do not adequately assess multiple intelligences. However, 
alternative methods have been less than convincing when their 
reliability are carefully scrutinized. Although they allow for 
the inclusion of evidence that standardized tests do not allow, 
they are susceptible to individual rater bias. 

Moreover, equity in educational opportunity is as important 
as identification of ability and achievement when identifying 
students for state academies. Educators have solid evidence that 
academic giftedness can be overlooked in conditions that are 
prevalent in many schools: poverty, prejudice, inferior programs. 
If state academies strive to identify the most qualified students, 
then their policies and procedures must allow for the social and 
cultural context of an applicant's record to be a meaningful part 
of the evaluation process. 



24 

ERiC ?7 



When the Indiana Academy for Science, Mathematics, and 
Humanities was planned, the admissions policies and procedures 
were designed with the explicit goals of identifying students who 
were characterized by their academic ability and achievement and 
creating a student body that represented the entire state. The 
resulting procedures utilized a holistic file review process that 
required the applications to be evaluated in the context of their 
local regions. This investigation was conducted as an evaluation 
of the initial admissions policies and procedures. Two questions 
were central. Did the procedures used by the academy achieve 
diversity in its student body? Second, did the holistic file 
review process show signs that it was both reliable and valid as a 
method for identifying students? 

The research procedures consisted of several stages. First, 
class profiles that summarized the demographic characteristics of 
the student body and their academic performance were reviewed. 
Next, statistical analyses were conducted in order to determine 
the validity of the academy's rating system. Canonical 
correlation was performed among the usual predictors of academic 
performance ( SAT scores, previous grades, and teacher 
recommendations) and the academy rating system (holistic scores 
for aptitude, achievement, accomplishment, and composite) . In 
addition, canonical correlation was performed among the usual 
predictors and academy GPA 1 s and the academy 1 5 holistic ratings 
and GPA 1 s . Finally, stepwise regression was performed in order to 
attempt to construct a model for predicting academic performance. 
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The results of the investigation allow the following general 
conclusions : 



1. The admissions policies and procedures of the 
Indiana Academy for Science, Mathematics, and 
Humanities used to select the inaugural class 
produced a student body that was characterized by 
academic excellence, geographic diversity, and 
ethnographic diversity for the state from which the 
applications were drawn . 

2. The aptitude, achievement, accomplishment, and 
composite ratings from the holistic file review 
process used by the Indiana Academy were positively 
correlated with the usual predictors of SAT scores, 
TSWE scores, grades, and teacher recommendations . 

3. The aptitude, achievement, accomplishment, and 
composite ratings from the holistic file review 
process used by the Indiana Academy were positively 
correlated with academy GPA's. 

4. Canonical correlation and stepwise regression 
procedures do not produce results that would lead to 
the construction of a useful analytical model for 
predicting academic performance in the academy. 

The general conclusion to be drawn from the investigation 

supports the continued use of the holistic file review process by 

the Indiana Academy for Science, Mathematics, and Humanities. The 

method allows flexibility for the social, cultural, and 

educational contexts that pertain to an individual's application 

to be considered, and it is proven to be valid in comparison to 

usual predictors that oftentimes do not permit for this 

flexibility. The holistic file review process used by the Indiana 

Academy for Science, Mathematics, and Humanities succeeds in 

demonstrating that policies designed to promote equity in 

educational opportunity are compatible with policies that demand 

educational excellence . 
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